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ORIGINAL ARTICLE 

Behavioral and neuroimaging evidence for overreliance on 
habit learning in alcohol-dependent patients 

Z Sjoerds^ S de Wit^'^ W van den Brink^ TW Robbins^ ATF Beekman\ BWJH Penninx^'^'^ and DJ Veltman^'^ 

Substance dependence is characterized by connpulsive drug-taking despite negative consequences. Animal research suggests an 
underlying imbalance between goal-directed and habitual action control with chronic drug use. However, this imbalance, and its 
associated neurophysiological mechanisms, has not yet been experimentally investigated in human drug abusers. The aim of the 
present study therefore was to assess the balance between goal-directed and habit-based learning and its neural correlates in 
abstinent alcohol-dependent (AD) patients. A total of 31 AD patients and 19 age, gender and education matched healthy controls 
(HQ underwent functional magnetic resonance imaging (fMRI) during completion of an instrumental learning task designed to 
study the balance between goal-directed and habit learning. Task performance and task-related blood oxygen level-dependent 
activations in the brain were compared between AD patients and healthy matched controls. Findings were additionally associated 
with duration and severity of alcohol dependence. The results of this study provide evidence for an overreliance on stimulus- 
response habit learning in AD compared with HC, which was accompanied by decreased engagement of brain areas implicated in 
goal-directed action (ventromedial prefrontal cortex and anterior putamen) and increased recruitment of brain areas implicated in 
habit learning (posterior putamen) in AD patients. In conclusion, this is the first human study to provide experimental evidence for a 
disturbed balance between goal-directed and habitual control by use of an instrumental learning task, and to directly implicate 
cortical dysfunction to overreliance on inflexible habits in AD patients. 
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INTRODUCTION 

Substance use disorders like alcohol dependence (AD) are 
regarded to be chronic, relapsing brain diseases.^"^ Although 
initial drug intake is associated with positive reinforcement and 
impulsive actions, chronic drug use is characterized by compulsive 
drug-taking behavior despite negative consequences.^"^ Animal 
research suggests that this shift from casual to compulsive drug 
use is based on increased habit formation at the expense of 
flexible, goal-directed action control.^"^° Yet, little is known about 
the mechanisms underlying this process in human drug abuse, 
and experimental evidence regarding the neuro-adaptations that 
develop during chronic alcohol use in humans is lacking. 

According to dual-systems accounts, two dissociable learning 
processes can be identified in instrumental behavior."'^^ On one 
hand, goal-directed actions are performed in order to achieve 
desirable goals (positive reinforcement) or to avoid undesirable 
outcomes (negative reinforcement). On the other hand, habitual 
responses are directly triggered by cues in the environment even 
when the outcomes have lost their goal value.^^ Studies in rodents 
have shown that the dorsomedial striatum and prelimbic cortex 
are involved in goal-directed actions.^^"^^ Imaging studies in 
healthy human volunteers indicate that goal-directed control is 
associated with activation of the ventromedial prefrontal cortex 



(VMPFC),^^'^^ an equivalent of the animal prelimbic cortex, and by 
the caudate head.^° In contrast, habit learning in rats is mediated 
by the dorsolateral striatum,^^'^^ whereas in humans it mainly 
involves the posterior putamen.^^ A recent diffusion tensor 
imaging study in healthy volunteers^^ showed that white matter 
connectivity within corticostriatal pathways is predictive of 
individual differences in the balance between goal-directed and 
habitual control: estimated tract strength between caudate and 
VMPFC predicted goal-directed action, and white matter tract 
strength between posterior putamen and premotor cortex 
predicted habitual behavioral control. 

Goal-directed actions are flexible but also slow; whereas, 
habitual responses are inflexible but have the advantage of being 
quick and automatic. Therefore, habit formation is a necessary and 
adaptive process. However, a gradual shift from goal-directed 
towards habitual control, mediated by phasic dopamine release in 
dorsolateral parts of the striatum,^^ has been hypothesized to 
underlie chronic compulsive drug-seeking in substance use 
disorders like AD (for reviews, see refs. 26,27). Indeed, long-term 
drug seeking in rodents is insensitive to decreasing outcome 
values, known as outcome-devaluation,^^'^^ suggesting habitual 
control. Lesion and microdialysis studies show that the formation 
of drug habits is associated with a shift away from engagement of 
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the prelimbic cortex^° and dorsomedial striatum^^ towards more 
dorsolateral parts of the striatum^ with prolonged drug use.^^ 
A dopamine-dependent cascading loop^^ from ventral to more 
dorsal parts of the striatum^^ is implicated in this course of habit 
formation. Reactivation of the hypoactivated prefrontal cortex in 
cocaine-dependent rats restores control over cocaine-seeking.^^ 
Inactivation of the dorsolateral striatum on the other hand 
reverses habitual drug seeking in rats that were exposed to 
prolonged alcohol^^ or cocaine use.^^ Thus, evidence suggests 
increased involvement of the dorsolateral striatum in rodents^ ^'^^"^^ 
and non-human primates^^ in long-term drug seeking. 

The animal studies reviewed above suggest that a progressive 
imbalance between goal-directed and habitual control is also 
likely to play a role in the development of compulsive drug habits 
in human substance abusers. To investigate this possibility, we 
examined the balance between goal-directed and habit learning, 
and its neural correlates, in abstinent AD patients and healthy 
controls (HC) during functional magnetic resonance imaging 
(fMRI) scanning, using an instrumental learning task developed to 
distinguish between goal-directed and habit-based learning. In 
addition, we examined whether this imbalance is dependent on 
disorder duration to test the theory that initial goal-directed 
actions are gradually replaced by stimulus-driven habitual 
behavior. We hypothesized stronger reliance on habits at the 
expense of goal-directed control in AD compared to HC, reflected 
in a relatively strong engagement of the neural habit pathway, 
comprising dorsolateral/posterior parts of the striatum (posterior 
putamen, caudate tail/body) and a relatively weak engagement of 
the goal-directed pathway in the VMPFC and dorsomedial/anterior 
parts of the striatum (caudate head, anterior putamen) during 
instrumental learning. Furthermore, this imbalance in behavior 
and its neural correlates was hypothesized to be more pro- 
nounced in longstanding as opposed to more recent-onset AD. 



MATERIALS AND METHODS 

Participants 

To facilitate a heterogeneous sample of AD participants with regard to the 
duration of the disorder, AD patients were recruited from two sources. One 
half of the group was recruited from addiction treatment clinics; the other 



AD patients and HC were recruited from the Netherlands Study of 
Depression and Anxiety (NESDA),^^ a large multi-site naturalistic cohort 
study with participants from primary care and outpatient mental health 
services. 

Forty-two patients were included with a current ( < 6 months) DSM-IV 
diagnosis of AD and no comorbid lifetime Axis-I diagnosis other than major 
depressive or anxiety disorders according to the Composite International 
Diagnostic Interview (CIDI).^^ In addition, a group of 20 HC was selected 
without any lifetime Axis-I disorder. All participants were free of major 
internal or neurologic disorders and MRI contraindications, did not use 
psychotropic medication other than a stable use of selective serotonin 
reuptake inhibitors or infrequent benzodiazepine use, and were free of 
current substance use disorders (other than alcohol for the AD group and 
smoking). All participants were asked to abstain from alcohol at least 24 h 
and from caffeine a few hours prior to the assessments. All participants had 
a confirmed breath alcohol level of 0.00% (Alcoscan Daisy-AL7000, Sentech, 
Korea). No participants scored higher than 8 points on the withdrawal 
symptom-checklist CIWA-Ar,^° and were therefore considered withdrawal- 
free. Mean abstinence duration of AD patients was 12 days (see Table 1). 

We excluded participants who: (1) had low quality imaging data (1 HC, 
2 AD); (2) responded on less than 90% of the trials during the instrumental 
learning task (2 AD); (3) showed performance below chance level during 
the training phase of the instrumental learning task (1 AD); and (4) urine- 
tested positive for drugs (cocaine) or benzodiazepines directly prior to the 
assessments (6 AD). The final dataset included 31 AD and 19 HC for 
analyses. 

Task paradigm 

A detailed task description can be found in the Supplementary Methods. 
Briefly, the instrumental learning task was developed to distinguish 
between goal-directed and habit-based learning,^^ and has been used 
successfully in studies to establish habits in both animals and humans.^"^'"^^""^"^ 
The task consists of a discrimination training phase and an outcome- 
devaluation test phase. 

During the discrimination training phase, participants learn by trial-and- 
error to respond (R) with a left or right button press to stimuli (S) in order 
to gain outcomes (O) that yield points representing monetary reward. 
Three trial types with different S-0 contingencies can be distinguished: 
standard, congruent and incongruent (see Figure 1). A core feature of the 
task is the differential involvement of goal-directed and habit learning 
systems during the different trial types. During standard trials different 
outcome-pictures follow the stimulus-pictures after a correct response, 
whereas during congruent trials the outcome-pictures are identical to the 
preceding stimulus-picture. Learning the correct response to each stimulus 



Table 1. Sample characteristics 




AD N = 31 


HC N=19 


Test statistic 


P-value 


Demographics 
















Age, M (s.d.) 


48.5 


8.5 


47.7 


10.6 


t-- 


= 0.26 


0.794 


Years of education, M (s.d.) 


13.5 


3.8 


13.7 


3.7 


U = 


310.00 


0.754 


Gender male, N (%) 


18 


58.1 


12 


63.2 




= 0.13 


0.721 


Right handed, N {%f 


28 


90.3 


16 


84.2 




= 0.42 


0.519 


Smokers, N (%)^ 


18 


58.1 


0 


0 




= 17.24 


<0.001 


Experienced stress^ 
















Before task, M (s.d.) 


43.9 


25.5 


33.4 


23.6 


U = 


220.50 


0.138 


After task, M (s.d.) 


37.4 


24.4 


28.3 


20.8 


t-- 


= 1.35 


0.184 


End of experimental day, M (s.d.) 


36.6 


26.9 


29.4 


25.2 


u= 


258.50 


0.271 


Clinical characteristics 
















IDS, M (s.d.)^ 


21.1 


11.1 


3.7 


4.1 


t-- 


= 7.85 


<0.001 


BAI, M (s.d.)^ 


12.4 


10.2 


2 


3 


u= 


= 75.00 


<0.001 


AUDIT M (s.d.)^ 


19.4 


7.7 


4.5 


2.7 


u= 


= 13.50 


<0.001 


Last alcoholic drink (days)^ 


12.3 


21.2 


5.0 


13.7 


u= 


202.50 


0.066 


AD duration, M (s.d.)^ 


15.0 


11.7 












Abbreviations: AD, alcohol dependent; AUDIT alcohol use disorder identification test; BAI, beck anxiety inventory; HC, healthy controls; IDS, inventory of 
depressive symptomatology; M, mean; s.d., standard deviation; f, student f-test; U, Mann-Whitney U-test;x^, chi-square test. ^Hand preference questionnaire.^" 
"^Fagerstrom.^^ "^Visual analog scale.^^ ^Inventory of depressive symptoms.^^ ^Beck anxiety inventory.^"^ ^Alcohol use disorder test.^^ ^First question of the 
CIWA-Ar. ^Duration calculated subtracting age of onset from age of current dependence, both assessed by the CIDI.^^ 
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Figure 1. Three trial types with stinnulus-response-outconne associations (left panel) and their involved learning systems (right panel). The 
standard and congruent discrinninations can be resolved with both the goal-directed and the habit systenn. In contrast, the incongruent 
discrinnination can only be resolved using the habit systenn. During instrunnental learning in the flexible goal-directed systenn, the outconne is 
represented in the associative structure, allowing incentive evaluation of the outconne to innpact on action selection. In the habit systenn, 
behavior is directly triggered by environnnental stinnuli via S-R associations, rendering action selection relatively efficient but also inflexible. 



during these two trial types can be established by using either the goal- 
directed S-O-R system or the habitual S-R system. Conversely, during 
incongruent trials, stimulus pictures in one trial are shown as outcomes in 
other trials and vice versa, that is, each picture functions as a stimulus and 
an outcome for opposing responses. In these trials, goal-directed learning 
is disadvantageous, and performance relies solely on habitual control 
via direct S-R associations. Indeed, previous studies have demonstrated 
that both humans and animals adopt an S-R habitual learning strategy 
to solve this incongruent trial type.^^'^^"^^ Contrasting blood oxygen level- 
dependent (BOLD)-signal of the different trial types during the discrimina- 
tion training phase allowed us to study neural correlates of the balance 
between the goal-directed and habit learning systems during instrumental 
learning (for the contrasts, see section Imaging Data Acquisition 
and Analysis). 

Following the discrimination training phase, an outcome-devaluation 
test phase assesses the strength of goal-directed R-0 associations. Here, 
some of the outcomes are devalued and participants have to use their 
knowledge of the response-outcome (R-0) relationships to (re)direct their 
choices towards still-valuable outcomes. Individuals who merely acquired 
knowledge about habitual S-R associations during the training phase have 
more difficulties to accurately complete the test phase than individuals 
whose instrumental learning was mainly based on goal-directed S-O-R 
associations. 

During the discrimination training phase and the outcome-devaluation 
test phase, we assessed the total percentage correct responses (task 
accuracy) and response times for each trial type separately. Learning 
effects over time were measured across the six equal blocks of the 
training phase. The task was assessed two times, once with neutral (fruit) 
pictures, and once with (simple) alcohol pictures (in counterbalanced 
order). However, picture-type did not yield any differences in behavioral or 



neuroimaging data. Therefore we took the two sessions together for the 
final analyses, in order to increase power. 

Statistical analyses 

Sample characteristics and task performance data were analyzed using 
SPSS 16.0. After testing for normality using Shapiro-Wilk's test, we used 
f-tests and whenever necessary non-parametric Mann-Whitney U or 

tests to examine group differences. Correlation analyses were performed 
using Spearman's p. 

Task accuracy and response times of the discrimination training phase 
and outcome-devaluation test phase were entered in a series of repeated 
measures ANOVAs with trial type (standard/congruent/incongruent) and 
(for the training phase) block as within-subject factors, and group as 
between-subjects factor. Post-iioc tests of significant main effects were 
analyzed using paired f-tests with Bonferroni correction for multiple 
comparisons. To test whether initial goal-directed control is gradually 
replaced by stimulus-driven habitual control in chronic AD, we assessed 
the association between AD duration and task performance in regression 
analyses within the AD group. 

All SPSS analyses were performed two-tailed with an alpha of 0.05, and 
all repeated-measure factors are reported with Greenhouse-Geisser 
sphericity corrections (Pgg). 

Innaging data acquisition and analysis 

Functional magnetic resonance imaging was performed at the Academic 
Medical Centre in Amsterdam using a 3T Philips Intera full-body MR-system 
(Philips Medical Systems, Best, The Netherlands) with a phased array SENSE 
RF 8-channel head coil. Functional BOLD signals were sequentially 
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Figure 2. Masks used for areas of interest, derived fronn the autonnatic anatonnical labelling atlas in WFU Pickatlas. Yellow: 'frontal_sup_nnediar 
& 'frontal_nned_orb' covering the nnedial prefrontal cortex, and a defined delineation at z < 25 for ventral orientation. Red: caudate & 
putannen, covering the striatunn. 



acquired with a T2*-weighted, gradient-echo planar imaging sequence and 
an estimated time-course series of approximately 360 volumes (self-paced) 
per session (TR/TE = 2300 ms/20 ms; matrix size = 96 x 95; voxel size = 2.29 
X 2.29 X 2.50 mm^• slices = 45). Imaging parameters were optimized 
according to the methods of Deichmann and colleagues^^ to minimize 
susceptibility and distortion artifacts in the orbitofrontal cortex: each 
volume was scanned with an orientation of 30° from Anterior-Posterior 
Commissure line. A T1 -weighted structural scan was made for anatomical 
reference with the fMRI data (TR/TE = 9 ms/3.6 ms; matrix size = 256 x 231; 
voxel size =1x1x1 mm^- slices = 1 70). 

Imaging analyses were performed using SPIV18 (Wellcome Trust Centre 
for Neuroimaging, London, UK). Images were manually reoriented to the 
Anterior-Posterior Commissure line, slicetimed, realigned, warped to 
Montreal Neurological Institute (MNI) space, and smoothed using a 
Gaussian kernel of 8 mm full-width-at-half-maximum. To remove low- 
frequency signal drift a high-pass filter (128 Hz) was applied. 

Statistical analysis of imaging data was performed using the General 
Linear Model.^^ Trial onsets were modeled with a delta function convolved 
with a synthetic hemodynamic response function and modulated for 
stimulus length (2000 ms). Obtained parameter estimates of the three trial 
types from the discrimination training phase were entered in a group*trial 
type full-factorial second-level (random effects) mixed-ANOVA model. 

During second-level analyses, we first assessed instrumental learning 
irrespective of the involvement of goal-directed or habit system, by 
examining BOLD-responses during all three trial types (standard, 
congruent and incongruent) contrasted against an implicit baseline. 
Second, to specifically assess the neural correlates of goal-directed 
learning, we contrasted the standard and congruent trials (both involving 
S-O-R goal-directed and S-R habit learning) against the incongruent trials 
(only involving S-R habit learning) according to the contrast: ((standard 
-i-congruent)>incongruent). Third, since the incongruent trials rely on S-R 
learning only, we compared the groups on these trial types in order to 
examine the involvement of brain pathways associated with S-R habit 
learning. To investigate the neural associations of instrumental learning 
with duration of AD, first level parameter estimates for these above- 
mentioned three contrasts of interest were entered into separate second- 
level regression analyses with duration of AD as predictor variable within 
the AD group. 

All imaging analyses were performed whole-brain, but we focused on 
areas we found in the analyses that are known to be involved in goal- 
directed and habit learning:^ ^"^^'^^'^"^'"^^ VMPFC, anterior putamen and 
caudate head for goal-directed learning, and posterior putamen and 
caudate tail/body for habit learning. Anterior versus posterior putamen 
was demarcated at MNI-coordinate y=2 (anterior>2; posterior < 2), in 
accordance with a previous connectivity study using the same task.^^ The 
mask for the VMPFC and striatum (caudate and putamen) was derived 
from the automatic anatomical labelling atlas incorporated in the WFU 
PickAtlas Tool v2.5.2'^^ (Figure 2). 

Main effects are reported at significance P<0.05 whole-brain family- 
wise error (FWE) corrected (Pfwe) for multiple comparisons. Group compari- 
sons and regression analyses with duration of AD were examined at 
P < 0.005 uncorrected (Puncorr.) with a cluster-extent of 5 voxels to reduce 
the risk of Type-ll error. To further protect against Type-I error, group 
comparisons were corrected for multiple comparisons on the voxel-level 
by using FWE-correction as implemented in SPM within the masks of 
interest, using a small-volume correction^^ by centering a lOmm-radius 
sphere around peak coordinates of activated clusters, as corresponding to 



previous studies.^^'^°'^^ The resulting volumes of interest had to meet 
P<0.05 FWE small-volume corrected (Pfwe_svc)/ to be considered 
significant. 



RESULTS 

Sannple characteristics 

For detailed infornnation on sample characteristics, see Table 1. 
Briefly, groups were nnatched on age, gender, education, hand- 
edness and experienced stress. The AD group contained smokers, 
whereas the HC group did not, and scored higher on alcohol use 
as well as depression and anxiety severity compared to the HCs. 

Task performance 

Discrimination training. Overall, participants showed the 
expected learning effect (F4,io7 = 103.695; Pgg < 0.001) and learned 
to respond on all trial types above chance level during the final 
block (all P< 0.005). In line with previous studies a main effect of 
trial type was observed (Fi^gei =8.155; Pgg < 0.005): performance 
on incongruent trials was inferior compared to congruent trials 
(Fbonf = 0.001) and marginally inferior compared to standard trials 
(Fbonf = 0.072), whereas performance on congruent and standard 
trials did not significantly differ (Pbonf = 0.221). For a plot depicting 
learning during discrimination training, the reader is referred to 
Supplementary Figure 52. No differences in performance between 
AD and HC were observed, indicating that exposure to the S-R-0 
contingencies was similar in both groups. 

Response times became faster as a consequence of training 
(^3,1 78 = 62.41; Pgg<0.005). A main effect of trial type (Fi,923 = 
24.57; Pgg<0.005) on response times was found with longer 
response times for the incongruent compared to the congruent 
and standard trials, and for the standard trials compared to the 
congruent trials. There was no speed-accuracy trade-off. 

Outcome-devaluation test. During the outcome-devaluation 
test phase, a main effect of trial type was found (Fi,63 = 
37.412; P<0.005): performance on the incongruent trials was 
inferior to the other two trial types (Pbonf<0.005 for both 
comparisons), and performance on the standard trials was inferior 
to the congruent trials (Pbonf = 0.005). Performance was signifi- 
cantly above chance level on congruent and standard trials (for 
both groups) (all F<0.001), but not on incongruent trials, 
indicating that discriminative actions on incongruent trials were 
exclusively mediated by habitual S-R associations, whereas 
actions on standard and congruent trials additionally benefited 
from the goal-directed system encoding the R-0 relationships. 

Importantly, a significant main effect for group regardless of 
trial type was found for outcome-devaluation (Fi 43 = 6.07; P = 
0.017) with poor performance in the AD group compared to HCs, 
indicating impaired R-0 knowledge for goal-directed action in AD. 
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Figure 3. Group connparisons of BOLD response during the instrunnental learning task. Displayed at P < 0.005 whole-brain uncorrected, extent 
threshold >5 voxels: (I) Group connparisons during instrunnental learning in general; (a) larger involvennent of the VMPFC (7 = 3.67) in HC 
connpared to AD; (b) larger involvennent of the posterior putannen (7 = 3.22) in AD connpared to HC. (II) Group connparisons during goal- 
directed learning: larger involvement of the VMPFC (7=3.45) and anterior putannen (7=3.63) in HC connpared to AD. (Ill) Group connparisons 
during S-R habit learning: larger involvennent of the posterior putannen (7 = 3.37) in AD connpared to HC. Abbreviations: AD, alcohol 
dependent; BOLD, blood oxygen level-dependent; HC, healthy controls; VMPFC, ventronnedial prefrontal cortex; Z, standard score Z-value. 



For a graphic representation of these findings the reader is 
referred to Supplementary Figure S3. 



Imaging results 

Instrumental learning (All trial types). This analysis revealed that 
the posterior putamen (7=6.38; Pfwe<0.05) and dorsal caudate 
nucleus (7=5.68; Pfwe < 0-05) were activated during instrumental 
learning (main effect). Group comparisons showed that during 



instrumental learning, the posterior putamen was more active in 
AD than in HC (7=3.22; PpwE-svc = 0.044). Furthermore, although 
the main effect did not reveal VMPFC involvement, HC did activate 
the VMPFC more than AD (7=3.67; Pfwe-svc = 0.012) (Figure 3-1, 
panels b and a, respectively). 

Goal-directed learning [(Standard-\-Congruent)>lncongruent]. The 
main effect of goal-directed learning showed VMPFC involve- 
ment (7>6; PpwE < 0.05). Moreover, this activation of the VMPFC 
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Figure 4. Regression of VMPFC activity with duration of alcohol dependence. Displayed at P<0.005 whole brain uncorrected, extent 
threshold >5 voxels. Scatterplot shows a negative correlation between parameter estimates in the VMPFC and duration of AD. Correlation 
coefficient is meant for visualization purposes only. AD, alcohol dependent; VMPFC, ventromedial prefrontal cortex. 



was significantly more pronounced in HC than in AD 
(Z=3.45; Pfwe-svc = 0-023). In addition, HCs showed significantly 
stronger activation of the anterior putamen compared to AD 
(7=3.63; Pfwe-svc = 0.013) (Figure 3-11). The AD group showed no 
regions with greater activation compared to the HCs. 

Habit learning (Incongruent trials). The main effect of S-R habit 
learning showed activation of the posterior putamen (7=7.48; 
^FWE < 0.05) and the dorsal caudate nucleus (7=4.95; Pfwe 

< 0.05). 

There was more activation in the posterior putamen (7=3.37; PpwE- 
svc = 0.029) in AD compared to HC (Figure 3-III). The HCs did not show 
increased activation in the areas of interest compared to AD. 

Post-hoc. Since AD and HC significantly differed in smoking and 
depression/anxiety symptoms, we added these variables as 
covariates in a post-hoc mixed-ANOVA model for behavioral data 
as well as BOLD response. The reported group differences 
remained significant. 

Associations with AD duration 

AD duration was not correlated with task performance in the 
discrimination training phase or the outcome-devaluation phase. 
Regression analysis of AD duration with brain activity during the 
goal-directed learning contrast [(Standard+Congruent)>lncongruent] 
showed that the VMPFC was less active in AD patients with a 
longer duration of AD (7=2.97; Puncorr. < 0.005) (Figure 4). After 
correcting for multiple comparisons, this cluster became margin- 
ally significant (p = - 0.569; Pfwe-svc = 0.107). Since AD duration 
was correlated with age (p = 0.486; P<0.01) we post-hoc 
performed multiple regression and partial correlation analyses 
in SPM, revealing that VMPFC activation was inversely correlated 
with AD duration, but not with age (data not shown). 



DISCUSSION 

To the best of our knowledge, this is the first study providing 
behavioral as well as neurophysiological evidence for an 
imbalance between goal-directed and habitual control in humans 
with a substance use disorder. We demonstrate that AD affects the 
ability to base instrumental actions on goal-directed R-0 knowl- 
edge. This effect was not due to differences in exposure to the 
S-R-0 contingencies, since AD patients and HCs performed 
equally well during a training phase. These results are therefore 
in line with the hypothesis of impaired goal-directed action 
and subsequent overreliance on S-R based habit learning in AD. 
Furthermore, we demonstrate that this imbalance is associated 
with decreased activity in the VMPFC and anterior putamen, brain 
regions previously implicated in goal-directed learning,^^"^° and 
increased activity in the posterior putamen, a region involved in 
habit learning.^^ 

A similar imbalance between goal-directed and habitual control 
has previously been shown to underlie several other psycho- 
pathological conditions associated with alterations in dopaminer- 
gic transmission. For example, patients with Parkinson's disease 
show an imbalance between goal-directed and habit learn- 
ing.^"^ The involvement of dopamine in goal-directed action is 
further supported by a study using an experimental dopamine 
depletion paradigm."^^ Moreover, patients with an obsessive- 
compulsive disorder exhibit a similar imbalance,^^ suggesting 
a shared mechanism across different forms of compulsive 
behavior." Finally, impulsivity, a trait repeatedly found in 
substance use disorders, is associated with excessive reliance on 
S-R habits at the expense of goal-directed behavior.^^ 

Two studies provide indirect evidence for an imbalance 
between goal-directed and habitual pathways in humans with 
substance use disorders. A recent fMRI study using a cue-reactivity 
paradigm^"^ showed that heavy drinkers (some with AD) showed 
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significantly higher cue-induced activation of the dorsal striatum, 
whereas light social drinkers showed higher cue-induced activa- 
tions in prefrontal areas and the ventral striatum. The authors 
interpreted these findings as an indication for a shift from ventral 
to dorsal striatal involvement in the course of a developing 
alcohol use disorder; a shift that is thought to be associated with 
the increasing role of habit-like drug seeking behavior. In a 
pharmacological fMRI study, compulsivity and its neural correlates 
were assessed using a reversal learning task.^^ Stimulant depen- 
dent subjects made significantly more perseverative responses 
than healthy volunteers, an effect that was negatively correlated 
with activation in the right caudate nucleus and normalized after 
administration of the dopamine D2/3 agonist pramipexole. It 
should be noted, however, that in contrast to the present study, 
these previous studies did not specifically tap into the neural 
correlates of goal-directed or habit learning. 

Although animal studies allow experimental initiation of 
substance dependence to track the gradual formation of 
substance-specific habits, ethical and practical issues evidently 
preclude such investigations of drug-specific habits in humans. 
Therefore, in the present study we took the approach of studying 
instrumental learning in the context of alcohol-related pictures 
versus neutral (fruit) pictures, with the aim of experimentally 
isolating alcohol-specific versus general habits. However, we 
found no significant differences between these stimulus types in 
performance or imaging data, suggesting a general shift towards 
habitual behavior, which is expressed even in the context of 
AD-irrelevant stimuli. 

This observed general shift towards S-R habits could be a 
consequence of chronic alcohol use by repeated exposure to the 
harmful effects of drugs of abuse^^'^^'^^ or a predisposing trait or 
vulnerability for the development of compulsive and impulse 
regulation disorders, or both.^^'^^ A pre-existing habitual trait 
could increase the risk of substance abuse and dependence after 
initial drug intake, but persistent substance use could also 
exacerbate the habitual trait already present, thus rendering users 
more prone to develop chronic and habitual drug-seeking 
behaviors. In this way, a vicious cycle is initiated, ultimately 
spiraling into clinical dependence.^° The current study reports a 
negative correlation between AD duration and engagement of the 
goal-directed pathway (VMPFC) during instrumental learning, 
suggesting that persistent substance abuse could play at least 
some role in the excessive reliance on habitual learning in AD 
patients. However, the reported negative association between 
goal-directed VMPFC activity and AD duration was subthreshold 
only, and should therefore be interpreted with caution. In 
addition, with the current crossectional study design, it is not 
possible to determine whether unbalanced goal-directed and 
habitual control systems are a consequence of detrimental alcohol 
use, or whether they reflect a premorbid individual difference 
which results in an increased risk for developing alcohol 
dependence. Prospective or longitudinal studies using similar 
methods as those used here are needed to assess whether an 
individual's degree of reliance on habitual control reflects a 
consequence of dependence, or is a risk factor in the develop- 
ment of early and chronic substance dependence. 

We were the first to use an instrumental learning task like the 
one designed by deWit et aC^ in an addicted patient group, in 
combination with fMRI. This study exemplifies an approach that is 
still exploratory and requires further development for these 
preliminary data to be considered definitive. However, the data 
are nevertheless consistent with a coherent theoretical approach 
derived from translational studies indicating an important role for 
habit-based behavior in stimulant drug addiction, now applied to 
human alcohol dependence. Therefore, the current findings 
contribute to the conceptualization of addiction as an overreliance 
on stimulus-driven habits at the expense of flexible, goal-directed 
action,^ leading to frequent and persistent substance use despite 
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serious negative consequences. To date, treatment programs for 
AD have mainly tried to reduce the positive reinforcing properties 
of drugs ('blocking the buzz' for example, with the opioid 
antagonist naltrexone) or to reduce the negative reinforcing 
aspects of chronic alcohol use represented by relief craving 
('curing the blues' for example, with acamprosate).^ While these 
outcome reevaluation interventions are of crucial importance to 
adjust the goal status of addictive substances, recent research has 
provided evidence that addictive stimulus-driven behavior may 
still persist.^^ Therefore, the findings of the current study suggest 
that new pharmacological or psychotherapeutic interventions 
should be developed to target inflexible, habitual drug-seeking. 

CONFLICT OF INTEREST 

The authors declare no conflict of interest. 



ACKNOWLEDGMENTS 

The VU University Medical Center Ethical Review Board approved this study, and 
written informed consent according to the declaration of Helsinki was obtained from 
all participants prior to study. This study was supported by grant 31 160004 from the 
Netherlands Organization for Health Research and Development (Zon-Mw). We thank 
the local treatment clinics Jellinek, De Maliebaan and the Home Clinic for recruitment 
of alcohol-dependent patients, and the Netherlands Study of Depression and Anxiety 
(NESDA) for the recruitment of alcohol-dependent patients and healthy controls. We 
are grateful to all participants who gave their time and support to participate in 
the study. 



REFERENCES 

1 McLellan AT, Lewis DC, O'Brien CP, Kleber HD. Drug dependence, a chronic 
medical illness: implications for treatment, insurance, and outcomes evaluation. 
JAMA 2000; 284: 1689-1695. 

2 Leshner Al. Addiction is a brain disease, and it matters. Science 1997; 278: 45-47. 

3 O'Brien CP, McLellan TA. Myths about the treatment of addiction. Lancet 1996; 
347: 237-240. 

4 American Psychiatric Association. Diagnostic and Statistical Manual of Mental 
Disorders. 4th edn. American Psychiatric Publishing: Washington DC, USA, 2001. 

5 Heilig M, Thorsell A, Sommer WH, Hansson AC, Ramchandani VA, George DT et al. 
Translating the neuroscience of alcoholism into clinical treatments: from blocking 
the buzz to curing the blues. Neurosci Biobehav Rev 2010; 35: 334-344. 

6 Koob GF, Volkow ND. Neurocircuitry of addiction. Neuropsychopharmacology 
2010; 35: 217-238. 

7 Ostlund SB, Balleine BW. On habits and addiction: an associative analysis of 
compulsive drug seeking. Drug Discov Today Dis Models 2008; 5: 235-245. 

8 Vanderschuren LJMJ, Everitt BJ. Drug seeking becomes compulsive after pro- 
longed cocaine self-administration. Science 2004; 305: 1017-1019. 

9 Everitt BJ, Robbins TW. Neural systems of reinforcement for drug addiction: from 
actions to habits to compulsion. Nat Neurosci 2005; 8: 1481-1489. 

10 Everitt BJ, Belin D, Economidou D, Pelloux Y, Dalley JW, Robbins TW. Review. 
Neural mechanisms underlying the vulnerability to develop compulsive drug-seeking 
habits and addiction. Philos Trans R Soc Lond B Biol Sci 2008; 363: 3125-3135. 

1 1 de Wit S, Dickinson A. Associative theories of goal-directed behaviour: a case for 
animal-human translational models. Psychol Res 2009; 73: 463-476. 

12 Dickinson A, Balleine BW. Motivational control of goal-directed action. Learn 
Behav 1994; 22: 1-18. 

13 Balleine BW, Dickinson A. Goal-directed instrumental action: contingency and 
incentive learning and their cortical substrates. Neuropharmacology 1998; 37: 
407-419. 

14 Yin HH, Knowlton BJ, Balleine BW. Blockade of NMDA receptors in the 
dorsomedial striatum prevents action-outcome learning in instrumental 
conditioning. Eur J Neurosci 2005; 22: 505-512. 

15 Yin HH, Ostlund SB, Knowlton BJ, Balleine BW. The role of the dorsomedial 
striatum in instrumental conditioning. Eur J Neurosci 2005; 22: 513-523. 

16 Corbit LH, Balleine BW. The role of prelimbic cortex in instrumental conditioning. 
Behav Brain Res 2003; 146: 145-157. 

17 Killcross S, Coutureau E. Coordination of actions and habits in the medial 
prefrontal cortex of rats. Cereb Cortex 2003; 13: 400-408. 

18 Valentin W, Dickinson A, O'Doherty JP. Determining the neural substrates of goal- 
directed learning in the human brain. J Neurosci 2007; 27: 4019-4026. 



© 2013 Macmillan Publishers Limited 



Translational Psychiatry (2013), 1 -8 



Habit learning in alcohol-dependent patients 
Z Sjoerds et al 



19 deWit S, Corlett PR, Aitken MR, Dickinson A, Fletcher PC. Differential engagement 
of the ventromedial prefrontal cortex by goal-directed and habitual behavior 
toward food pictures in humans. J Neurosci 2009; 29: 11330-11338. 

20 Tanaka SC, Balleine BW, O'Doherty JP. Calculating consequences: brain systems 
that encode the causal effects of actions. J Neurosci 2008; 28: 6750-6755. 

21 Yin HH, Knowlton BJ, Balleine BW. Lesions of dorsolateral striatum preserve 
outcome expectancy but disrupt habit formation in instrumental learning. Eur J 
Neurosci 2004; 19: 181-189. 

22 Packard MG, Knowlton BJ. Learning and memory functions of the basal ganglia. 
Annu Rev Neurosci 2002; 25: 563-593. 

23 Tricomi E, Balleine BW, O'Doherty JP. A specific role for posterior dorsolateral 
striatum in human habit learning. Eur J Neurosci 2009; 29: 2225-2232. 

24 de Wit S, Watson P, Harsay HA, Cohen MX, van de VI, Ridderinkhof KR. Corti- 
costriatal connectivity underlies individual differences in the balance between 
habitual and goal-directed action control. J Neurosci 2012; 32: 12066-12075. 

25 Kienast T, Heinz A. Dopamine and the diseased brain. CNS Neurol Disord Drug 
Targets 2006; 5: 109-131. 

26 Newlin DB, Strubler KA. The habitual brain: an "adapted habit" theory of 
substance use disorders. Subst Use Misuse 2007; 42: 503-526. 

27 Lesscher HM, Vanderschuren LJ. Compulsive drug use and its neural substrates. 
Rev Neurosci 2012; 23: 731-745. 

28 Dickinson A, Wood N, Smith JW. Alcohol seeking by rats: action or habit? Q J Exp 
Psychol B 2002; 55: 331-348. 

29 Miles FJ, Everitt BJ, Dickinson A. Oral cocaine seeking by rats: action or habit? 
Behav Neurosci 2003; 117: 927-938. 

30 Chen BT, Yau HJ, Hatch C, Kusumoto-Yoshida I, Cho SL, Hopf FW et al. Rescuing 
cocaine-induced prefrontal cortex hypoactivity prevents compulsive cocaine 
seeking. Nature 2013; 496: 359-362. 

31 Corbit LH, Nie H, Janak PH. Habitual alcohol seeking: time course and the con- 
tribution of subregions of the dorsal striatum. Biol Psychiatry 2012; 72: 389-395. 

32 Zapata A, Minney VL, Shippenberg TS. Shift from goal-directed to habitual cocaine 
seeking after prolonged experience in rats. J Neurosci 2010; 30: 15457-15463. 

33 Haber SN, Fudge JL, McFarland NR. Striatonigrostriatal pathways in primates form 
an ascending spiral from the shell to the dorsolateral striatum. J Neurosci 2000; 20: 
2369-2382. 

34 Jonkman S, Pelloux Y, Everitt BJ. Differential roles of the dorsolateral and mid- 
lateral striatum in punished cocaine seeking. J Neurosci 2012; 32: 4645-4650. 

35 Belin D, Everitt BJ. Cocaine seeking habits depend upon dopamine-dependent 
serial connectivity linking the ventral with the dorsal striatum. Neuron 2008; 57: 
432-441 . 

36 Vanderschuren LJMJ, Di Ciano P, Everitt BJ. Involvement of the dorsal striatum in 
cue-controlled cocaine seeking. J Neurosci 2005; 25: 8665-8670. 

37 Porrino LJ, Lyons D, Smith HR, Daunais JB, Nader MA. Cocaine self-administration 
produces a progressive involvement of limbic, association, and sensorimotor 
striatal domains. J Neurosci 2004; 24: 3554-3562. 

38 Penninx BWJH, Beekman ATF, Smit JH, Zitman FG, Nolen WA, Spinhoven P et al. 
The Netherlands study of depression and anxiety (NESDA): rationale, objectives 
and methods. Int J Methods Psychiatr Res 2008; 17: 121-140. 

39 Robins LN, Wing J, Wittchen HU, Helzer JE, Babor TF, Burke J et al. The composite 
international diagnostic interview. An epidemiologic instrument suitable for 
use in conjunction with different diagnostic systems and in different cultures. 
Arch Gen Psychiatry 1988; 45: 1069-1077. 

40 Sullivan JT, Sykora K, Schneiderman J, Naranjo CA, Sellers EM. Assessment of 
alcohol withdrawal: the revised clinical institute withdrawal assessment for 
alcohol scale (CIWA-Ar). Br J Addict 1989; 84: 1353-1357. 

41 de Wit S, Niry D, Wariyar R, Aitken MRF, Dickinson A. Stimulus-outcome inter- 
actions during instrumental discrimination learning by rats and humans. J Exp 
Psychol Anim Behav Process 2007; 33: 1-11. 

42 de Wit S, Barker RA, Dickinson A, Cools R. Habitual versus goal-directed action 
control in Parkinson disease. J Cogn Neurosci 2011; 23: 1218-1229. 

43 de Wit S, Standing HR, Devito EE, Robinson OJ, Ridderinkhof KR, Robbins TW et al. 
Reliance on habits at the expense of goal-directed control following dopamine 
precursor depletion. Psychopharmacology (Berl) 2012; 219: 621-631. 



44 Gillan CM, Papmeyer M, Morein-Zamir S, Sahakian BJ, Fineberg NA, Robbins TW 
et al. Disruption in the balance between goal-directed behavior and 
habit learning in obsessive-compulsive disorder. Am J Psychiatry 2011; 168: 
718-726. 

45 Deichmann R, Gottfried JA, Hutton C, Turner R. Optimized EPI for fMRI studies of 
the orbitofrontal cortex. Neuroimage 2003; 19(Pt 1): 430-441. 

46 Friston KJ, Holmes AP, Poline JB, Grasby PJ, Williams SC, Frackowiak RS et al. 
Analysis of fMRI time-series revisited. Neuroimage 1995; 2: 45-53. 

47 Yin HH, Knowlton BJ. The role of the basal ganglia in habit formation. Nat Rev 
Neurosci 2006; 7: 464-476. 

48 Maldjian JA, Laurienti PJ, Kraft RA, Burdette JH. An automated method for 
neuroanatomic and cytoarchitectonic atlas-based interrogation of fMRI data sets. 
Neuroimage 2003; 19: 1233-1239. 

49 Worsley KJ, Marrett S, Neelin P, Vandal AC, Friston KJ, Evans AC. A unified 
statistical approach for determining significant signals in images of cerebral 
activation. Hum Brain Mapp 1996; 4: 58-73. 

50 van Tol MJ, van der Wee NJA, Demenescu LR, Nielen MM, Aleman A, Renken R 
et al. Functional MRI correlates of visuospatial planning in out-patient depression 
and anxiety. Acta Psychiatr Scand 2011; 124: 273-284. 

51 van Hoist RJ, van Holstein M, van den Brink W, Veltman DJ, Goudriaan AE. 
Response inhibition during cue reactivity in problem gamblers: an fMRI study. 
PLoS One 2012; 7: e30909. 

52 Fontenelle LF, Oostermeijer S, Harrison BJ, Pantelis C, Yucel M. Obsessive- 
compulsive disorder, impulse control disorders and drug addiction: common 
features and potential treatments. Drugs 2011; 71: 827-840. 

53 Hogarth L, Chase HW, Baess K. Impaired goal-directed behavioural control in 
human impulsivity. Q J Exp Psychol B 2012; 65: 305-316. 

54 Vollstadt-Klein S, Wichert S, Rabinstein J, Buhler M, Klein O, Ende G et al. Initial, 
habitual and compulsive alcohol use is characterized by a shift of cue processing 
from ventral to dorsal striatum. Addiction 2010; 105: 1741-1749. 

55 Ersche KD, Roiser JP, Abbott S, Craig KJ, Muller U, Suckling J et al. Response 
perseveration in stimulant dependence is associated with striatal dysfunction and 
can be ameliorated by a D(2/3) receptor agonist. Biol Psychiatry 201 1; 70: 754-762. 

56 Nelson A, Killcross S. Amphetamine exposure enhances habit formation. 
J Neurosci 2006; 26: 3805-381 2. 

57 Belin D, Mar AC, Dalley JW, Robbins TW, Everitt BJ. High impulsivity predicts the 
switch to compulsive cocaine-taking. Science 2008; 320: 1352-1355. 

58 Hogarth L, Chase HW. Parallel goal-directed and habitual control of human drug- 
seeking: implications for dependence vulnerability. J Exp Psychol Anim Behav 
Process 2011; 37: 261-276. 

59 Hogarth L. Goal-directed and transfer-cue-elicited drug-seeking are dissociated by 
pharmacotherapy: evidence for independent additive controllers. J Exp Psychol 
Anim Behav Process 201 2; 38: 266-278. 

60 Van Strien JW. Classificatie van links- en rechtshandige proefpersonen [Classifi- 
cation of left- and right-handed research participants]. Ned Tijdschr Psychol 1992; 
47: 88-92. 

61 Heatherton TF, Kozlowski LT, Frecker RC, Fagerstrom KO. The Fagerstrom test 
for nicotine dependence: a revision of the Fagerstrom tolerance questionnaire. 
Br J Addict 1 991 ; 86: 1 1 1 9-1 1 27. 

62 Schwabe L, Wolf OT. Stress prompts habit behavior in humans. J Neurosci 2009; 
29: 7191-7198. 

63 Rush AJ, Gullion CM, Basco MR, Jarrett RB, Trivedi MH. The Inventory of Depressive 
Symptomatology (IDS): psychometric properties. Psychol Med 1996; 26: 477-486. 

64 Beck AT, Epstein N, Brown G, Steer RA. An inventory for measuring clinical anxiety: 
psychometric properties. J Consult Clin Psychol 1988; 56: 893-897. 

65 Babor TF, Kranzler HR, Lauerman RJ. Early detection of harmful alcohol 
consumption: comparison of clinical, laboratory, and self-report screening 
procedures. Addict Behav 1989; 14: 139-157. 



©0®© 



This work is licensed under a Creative Commons Attribution- 
NonCommercial-NoDerivs 3.0 Unported License. To view a copy of 



this license, visit http://creativecommons.Org/licenses/by-nc-nd/3.0/ 



Supplementary Information accompanies the paper on the Translational Psychiatry website (http://www.nature.com/tp) 



Translational Psychiatry (2013), 1 -8 



© 2013 Macmillan Publishers Limited 



